Resource allocation aware queuing of requests for media resources

ABSTRACT

Resource allocation aware processing of requests for media resources is disclosed. A queue is defined. A media resource is allocated to the queue. A media resource request is associated with the queue.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 10/737,170, entitled RESOURCE ALLOCATION AWARE QUEUING OFREQUESTS FOR MEDIA RESOURCES filed Dec. 16, 2003, which is incorporatedherein by reference for all purposes, and which claims priority to U.S.Provisional Application No. 60/434,471, entitled AUTOMATED MEDIAMANAGEMENT filed Dec. 18, 2002, which is incorporated herein byreference for all purposes.

FIELD OF THE INVENTION

The present invention relates generally to removable storage media. Morespecifically, resource allocation aware queuing of requests for mediaresources is disclosed.

BACKGROUND OF THE INVENTION

Fully or partially automated media libraries, sometimes referred to as“libraries” or “robots”, are available to store and manipulate removablestorage media, such as tapes used to store computer data for backup orarchive purposes. A typical library may be equipped with a robotic orother mechanism for manipulating the media stored therein, such as byinserting a selected volume or unit of the media (e.g., a particulartape) into a read/write device associated with the unit, e.g., a tapedrive configured to write data to and/or read data from the media. Inthe computer network environment, for example, one or more backupapplications may be used to store data from one or more computers orother devices connected to the network (sometimes referred to herein asnetwork “nodes” or “hosts”) on storage media associated with a library.

A media management application may be provided to facilitate thetracking of removable storage media resources and to coordinate theservicing of requests for removable media storage resources, such as arequest by a backup application that a particular volume of media bemounted on a particular drive for a backup (or restore) operation. Insome network environments, multiple competing demands for the sameresource may be received by the media management application at the sametime. For example a first request that a first volume be mounted on adesignated media storage drive may be received and a second request thata second volume be mounted on the same drive may received while thefirst request is still pending. In such situations, the media managementapplication must determine which request to service first.

One typical approach is to assign different priorities to differenthosts and/or data sets having different levels of importance to networkstakeholders (e.g., a commercial or other enterprise) and/or havingdifferent requirements for backup and/or restoration using removablestorage media and associated storage devices. However, such a use ofpriorities may not be sufficient to ensure that the most importantrequests are serviced in a timely manner. For example, competingrequests assigned the same priority may be received at the same time.Also, a storage device may be busy with a lower priority request thattakes a long time to complete at a time when an urgent, higher priorityrequest for the same device is received.

Therefore, there is a need for a way to ensure that removable storagemedia resources (e.g., drives) are utilized in a way that ensures thatthe varying removable storage media resource requirements of differenthosts and/or data sets are met.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating one exemplary embodiment of anetwork environment and an associated media management system.

FIG. 2 is a flow chart illustrating a process used in some embodimentsto provide resource allocation aware queuing of requests for mediaresources.

FIG. 3 is a flow chart illustrating a process used in some embodimentsto configure one or more queues, as in step 202 of FIG. 2.

FIG. 4 illustrates a set of queues defined via the process shown in FIG.3.

FIG. 5 is a flowchart illustrating a process used in some embodiments toreceive requests and assign them to a queue, as in step 204 of FIG. 2.

FIG. 6 is a flow chart illustrating a process used in some embodimentsto service requests in accordance with their assigned queue andpriority, as in step 206 of FIG. 2.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess, an apparatus, a system, a composition of matter, a computerreadable medium such as a computer readable storage medium or a computernetwork wherein program instructions are sent over optical or electroniccommunication links. In this specification, these implementations, orany other form that the invention may take, may be referred to astechniques. In general, the order of the steps of disclosed processesmay be altered within the scope of the invention.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example andinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Resource allocation aware queuing of requests for media resources isdisclosed. A media resource request queue is defined. One or moreremovable storage media resources, such as a read/write device, areassociated with the queue. A request category is associated with thequeue. Subsequently, removable storage media resource requests that areassociated with the request category are placed in the queue andserviced by a resource associated with the queue.

FIG. 1 is a block diagram illustrating one exemplary embodiment of anetwork environment and an associated media management system. Thesystem 100 comprises a network 102, which may be a local area network(LAN) or any type of private or public network. The system 100 furthercomprises servers A, B, C, and D, identified by reference numerals 104,106, 108, and 110, respectively, in FIG. 1, connected to network 102. Inthe example shown in FIG. 1, a first backup application, such as theNetWorker backup application available commercially from the LegatoSoftware Division of EMC Corporation, is installed on server A (104),and a second backup application is installed on server B (106). Thefirst and second backup applications may be the same or differentproducts. The data on server C (108) is backed up by both the firstbackup application installed on server A (104) and the second backupapplication installed on server B (106), as is indicated in FIG. 1 bythe letters “A” and “B” in parentheses below the letter “C”. Such aconfiguration may be used, e.g., to provide two independent backups forparticularly critical data. Server D (110) is backed up by the firstbackup application installed on server A (104). Server A may likewisecomprise a body of data that is backed up by operation of the firstbackup application installed on server A, and server B may comprise abody of data that is backed up by operation of the second backupapplication installed on server B. The storage media used by the firstand second backup applications installed on servers A and B,respectively, reside in two storage media libraries of different types.SCSI library 112 is a library configured to be controlled directly by alibrary host 114 via a small computer systems interface (SCSI)connection. Library host 114 is connected to SCSI library 112 and tonetwork 102. ACSLS library 116 is an automated cartridge system librarysoftware-controlled library of the type available commercially fromStorage Technology Corporation (StorageTek) of Louisville, Colo. AnACSLS-type library such as library 116 is controlled using a softwarecontroller provided for that purpose, as opposed to being controlleddirectly by the library host. Library host 118 is connected to andconfigured to control ACSLS library 116. Library host 118 also isconnected to network 102. While examples of a SCSI and ACSLS typelibrary are shown in FIG. 1, any number of combination of types oflibraries may be used, including without limitation IBM 3494, ADIC AML,and/or any other type of library. SCSI library 112 has associated withand connected to it tape drives 120, 122, and 124. Tape drive 120 isconnected to a network attached storage (NAS) device 126. The data onNAS 126 is backed up by operation of the second backup applicationinstalled on server B. NAS 126 also has a connection to network 102.ACSLS library 116 has associated with and connected to it tape drives128, 130, 132, and 134. Tape drive 128 is connected to server A. Tapedrives 130, 132, and 134 are connected to servers C (108) and D (110)via a storage area network (SAN) 136. SAN 136 makes it possible for eachof servers C and D to read from or write to any one of the SAN-connectedtape drives 130, 132, and 134.

A media management application is installed on a media manager 138 tocoordinate operations between the first backup application running onserver A and the second backup application running on server B, such asby receiving and arbitrating between potentially competing requests forresources associated with libraries 112 and 116, as well as executedsuch requests. For example, the media manager may receive requests fromthe backup applications that a particular tape residing in one of thelibraries be inserted into a tape drive associated with that library.The media management application may provide other functionality, suchas keeping track of tapes stored in the libraries and elsewhere. Themedia manager 138 has a connection to the network 102, which it uses tocommunicate with other nodes connected to network 102 as described morefully below. The media manager 138 may comprise a server connected tonetwork 102.

To ensure the availability of removable storage media resources (e.g.,devices, such as tape drives) to serve critical requests, such as torestore a mission critical system or particularly critical data, thetechniques described herein may be used to allocate or dedicate a subsetof the available removable storage media resources to a queueestablished to service requests associated with the critical systemand/or data. Absent such an allocation, a critical request may not beserviced in an adequately timely manner. Consider an example in whichthe data requiring backup in the network environment 100 of FIG. 1comprised legal department files stored on server A (104), source codestored on server B (106), a mission critical database (e.g., a salestransaction and order status database) stored on server C (108),accounting data stored on server D (110), and user files stored on NAS126.

If due to regulatory or other legal requirements a large number offiles, or a great deal of accounting data of various classes residing onserver D (110) were required to be backed up onto different volumes ofstorage media, absent an allocation of media resources the backupapplication running on server A (104) might request that three differenttapes be mounted on tape drives 130, 132, and 134 to back up accountingdata on server D (110). If while these three backup operations werebeing performed the need arose to restore mission critical data toserver C (108), no tape drive to which server C has a connection wouldbe available to service the request. Under such circumstances, even ifthe media manager were to assign the highest priority to the requestand/or otherwise ensure that the request related to restoring server Cwould be the next one serviced by the first tape drive to becomeavailable on SAN 136, an unacceptable amount of time might elapse beforesuch a resource became available.

To avoid the situation described above and similar potential problems,resource allocation aware queuing of requests for media resources isdisclosed. For example, in the case of the network environment 100 ofFIG. 1, a queue may be defined for the purpose of servicing requestswith respect to the database stored on server C. One or more of the tapedrives on SAN 136 may be allocated to this queue, reserving thedevice(s) so allocated for the servicing of requests associated withserver C. Using this approach, there is no risk that requests associatedwith server D, or any other host that may be on the SAN 136, may blockthe servicing of critical requests associated with server C.

FIG. 2 is a flow chart illustrating a process used in some embodimentsto provide resource allocation aware queuing of requests for mediaresources. In step 202, one or more queues are configured. Step 202 maycomprise defining one or more queues, assigning one or more storagedevices with each queue, defining one or more request categories (e.g.,a category for requests associated with the database on server C),and/or assigning one or more request categories to each queue. In step204, storage media resource requests, e.g., a request to mount aspecified volume of storage media on designated device, are received(e.g., by a media management application) and assigned to an appropriateplace in an appropriate queue. Step 204 is repeated as each request isreceived. In step 206, requests in the queue(s) are serviced inaccordance with their associated queue and priority. For example, arequest that based on its assigned priority, time of receipt, etc. isthe request next in line to be serviced in a queue associated with adrive that just became available (e.g., because a backup, restore, orother operation for which the device previously was being used wascompleted) may in an iteration of step 206 be serviced by mounting thevolume indicated in the request on the available drive. Step 206 isrepeated as each request is serviced.

FIG. 3 is a flow chart illustrating a process used in some embodimentsto configure one or more queues, as in step 202 of FIG. 2. In step 302,one or more data zones are defined. A data zone may comprise a set ofhosts and associated data served by a particular backup applicationprogram. For example, in the network environment 100 of FIG. 1, a firstdata zone may be defined for the hosts associated with a first backupapplication running on server A (i.e., servers A, C, and D) and a seconddata zone may be defined for the hosts associated with a second backupapplication running on server B (i.e., servers B and C and NAS 126). Insome embodiments, a data zone administrator may be designated and giventhe access required to configure queues with respect to his/herparticular data zone, as described more fully below. In step 304, one ormore queues are defined. Defining a queue may comprise receiving anindication from a user that a new queue should be set up and receivingfrom the user (or assigning) a name or other identifier for the queue.In step 306, one or more storage devices (e.g., tape drives) areassociated with each queue. Step 306 may comprise receiving from a uservia a user interface an indication that a designated tape drive shouldbe associated with a particular queue. In some embodiments, each device(e.g., drive) may be assigned to only one queue. Devices not assigned toany queue are in some embodiments assigned by default to a default queueused to service requests that are not associated with any other queue.In step 308, each queue is associated with one or more data zones. Step308 may comprise giving a data zone administrator the access required toassign request categories to a queue, as described more fully below. Instep 310, one or more request categories are defined. In someembodiments, the data zone administrator defines the request categoriesfor his/her data zone. For example, in the case of a data zoneassociated with a backup application that associates related resourcestogether in pools (e.g., related volumes of removable storage media),such as the NetWorker application described above, each pool may beassociated with a request category. Likewise, request categories may bedefined, such as through a user interface, by defining a requestcategory for a body of related data (e.g., accounting data, source code,user files, legal department files, etc.) and/or by defining a categoryby reference to a particular host (client), e.g., requests associatedwith server A in FIG. 1. In step 312, each request category isassociated with a queue.

FIG. 4 illustrates a set of queues defined via the process shown in FIG.3. The set of queues 400 comprises five queues 402, 404, 406, 408, and410, also identified in FIG. 4 as Q1-Q5. The queues 400 are set up asshown for purposes of illustration and use the network environment 100as shown in FIG. 1 by way of example. In the example shown, queues 406,408, and 410 have been assigned to a data zone A and queues 402, 404,and 406 have been assigned to a data zone B. In this example, data zoneA corresponds to the clients and storage nodes associated with the firstbackup application running on server A (104) of FIG. 1 and data zone Bcorresponds to the clients and storage nodes associated with the secondbackup application running on server B (106) of FIG. 1. Each of thequeues 400 has one or more storage devices allocated to it. In thisexample, device 120 is assigned queue Q1 (402), devices 122 and 124 areassigned to queue Q2 (404), devices 130 and 132 are assigned to queue Q3(406), device 128 is assigned to queue Q4 (408), and device 134 isassigned to queue Q5 (410).

Each queue is shown in FIG. 4 as having a single request categoryassociated with it. However, in other embodiments more than one requestcategory may be assigned to a queue. In the example shown, the requestcategories are based on the nature of the data that the backupapplications are configured to backup/restore on each network client.For example, queue Q1 (402) has assigned to it a request category named“user files”, which corresponds to the data backed up on the NAS 126.Queue Q2 (404) has assigned to it a request category named “sourcecode”, which corresponds to the data backed up on server B. Queue Q3(406) has assigned to it a request category named “database”, whichcorresponds to the data backed up on server C (by both the first andsecond backup applications, i.e., in both data zones A and B). Queue Q4(408) has assigned to it a request category named “legal”, whichcorresponds to the data backed up on server A. Finally, queue Q5 (410)has assigned to it a request category named “accounting”, whichcorresponds to the data backed up on server D.

In the example shown in FIG. 4, one can see from FIG. 1 that the queuesQ3 (406) and Q5 (410) operate to allocate two of the three tape driveson SAN 136 (i.e., drives 130 and 132) to supporting requests associatedwith the database stored on server C (108) and the third drive on SAN136 (i.e., drive 134) to supporting requests associated with theaccounting files associated with server D (110). In this way, theresources on SAN 136 can be allocated in the way deemed most appropriateby the responsible network administrators, ensuring that the resourcesare used and/or available as required to support the data management(e.g., backup and restore) of the enterprise or other stakeholder(s)associated with the network.

Each of the queues Q1-Q5 has associated with it a pending request area,identified in FIG. 4 as areas 412, 414, 416, 418, and 420, respectively.These areas are used as described more fully below to track pendingrequests associated with each queue. The queues themselves may beimplemented in any number of ways known to those of ordinary skill inthe art. For example, each may comprise a data structure for storingpending requests and related data (e.g., priority, place in line, orderor time in which received, etc.) and logic for receiving requests,determining their relative order within the queue, andselecting/identifying from among the pending request the next request tobe serviced when a resource (e.g., drive) associated with the queuebecomes available.

In the example shown in FIG. 4, all of the tape drives in the networkenvironment 100 of FIG. 1 have been assigned to a queue, and all of therequests that may be generated with respect to hosts (clients) innetwork environment 100 have been assigned to a request category, eachof which has in turn been assigned to a queue. However, in otherembodiments, a user may choose not to create a queue for each deviceand/or type of request, and to instead create one or more queues tohandle a subset of the requests associated with the network environment,such as to ensure that the most critical requests are serviced in atimely manner. For example, referring further to FIG. 4, in someembodiments it may be sufficient or advantageous to define only queuesQ3 and Q5, to allocate resources on the SAN 136 of FIG. 1. In such acase, requests not associated with a request category and/or queue willbe handled by a default queue, such as a default queue for the data zonefrom which the request originated.

FIG. 5 is a flowchart illustrating a process used in some embodiments toreceive requests and assign them to a queue, as in step 204 of FIG. 2.In step 502, a request for a removable storage media resource isreceived. The request may comprise, for example, a request that aparticular volume of storage media (e.g., a specified tape) be mountedon a particular device (e.g., tape drive). In step 504, the request isassociated with a request category and a data zone. In some embodiments,the request may indicate the request category and/or data zone to whichit relates. In some embodiments, the request may comprise otherinformation that may be used to associate the request with a requestcategory and/or data zone, such as information identifying a pool, aclient (host) system, or the data or data type to which the requestrelates, or indicating the backup or other application that generatedthe request. In some embodiments, not shown in FIG. 5, if no requestcategory is indicated (or the request category cannot be determinedbased on other information), the request is assigned to a default queuefor the data zone to which the request relates. In step 506, thepriority level of the request is determined. In some embodiments, step506 may comprise receiving with the request an indication of thepriority of the request. In some embodiments, the priority may bedetermined at least in part by the type of operation associated with therequest (e.g., a request associated with a restore operation may havehigher priority than a request associated with a backup operation). Insome embodiments, the priority may be determined at least in part basedon the data and/or host to which the request relates. In step 508, therequest is place in the appropriate place in line in the appropriatequeue. Step 508 may comprise placing the request in the queue thatcorresponds to the request category and data zone associated with therequest, as determined in step 504. Step 508 may comprise placing therequest in the queue in a place determined at least in part by thepriority determined in step 506. For example, a request may be placed inthe queue in a position ahead of pending requests having a lowerpriority and behind pending requests having higher priority. In someembodiments, requests of the same priority are handled on afirst-in-first-out (FIFO) basis. In some embodiments, once a request isplaced within a queue in the appropriate place a user may alter theposition of the request within the queue. A user interface may beprovided to facilitate such altering of the order within a queue, suchas by allowing a user to click on and drag a request from a currentposition in the queue to a new position.

FIG. 6 is a flow chart illustrating a process used in some embodimentsto service requests in accordance with their assigned queue andpriority, as in step 206 of FIG. 2. In step 602, an indication isreceived that a resource associated with a queue is available. Such anindication may comprise, e.g., an indication from the backup applicationthat a previously serviced request is complete. In other embodiments,the indication may comprise an indication received from the library thata tape dismount operation with respect to the drive in question has beencompleted. In step 604, the next request in line in the queue isidentified from among the requests pending in the queue. In someembodiments, the requests are stored in the queue in the order in whichthey are to be serviced, based on the nature of the request and/or otherpriority information, as described above. In such embodiments, step 604comprises reading from the first (i.e., next to be serviced) position inthe queue information about the next request due to be serviced. In step606, the request identified in step 604 is serviced, e.g., by causing atape specified in the request to be mounted on a drive specified in therequest (and/or a drive associated with the queue, if none is specifiedin the request).

Requests in a default queue may be handled similarly to requests inqueues to which resources have been allocated, except that the resources(e.g., drives) available to service such requests are those to which thehost associated with each respective request has a connection and thatare not allocated to another queue.

Using the techniques described herein, removable media storage resources(e.g., tape drives) may be allocated to one or more queues, dedicatingthem to servicing requests associated with the respective queue to whicheach device is assigned. This provides network administrators with theability to ensure that resources are used in an optimal manner and thatresources are available in an adequately timely manner to servicemission critical requests.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

1. A method for processing requests for media resources, comprising:defining a queue; allocating a media resource to the queue; andassociating a media resource request with the queue.
 2. The method ofclaim 1, wherein associating a media resource request with the queuecomprises associating the media resource request with a request categoryassociated with the queue.
 3. The method of claim 1, further comprisingdefining a request category; and associating the request category withthe queue; wherein associating a media resource request with the queuecomprises associating the media resource request with the requestcategory.
 4. The method of claim 1, wherein the media resource comprisesa media storage device.
 5. The method of claim 1, wherein the mediaresource comprises a tape drive.
 6. The method of claim 1, wherein themedia resource request comprises a request to mount a volume ofremovable storage media on the media resource.
 7. The method of claim 1,further comprising receiving the media resource request from a host. 8.The method of claim 1, further comprising receiving the media resourcerequest from a backup application.
 9. The method of claim 1, wherein themedia resource is allocated exclusively to the queue.
 10. The method ofclaim 1, wherein the queue comprises a first queue, the media resourcecomprises a first media resource, and the media resource requestcomprises a first media resource request, further comprising: receivinga second media resource request; determining that the second mediaresource request is not associated with the first queue; and associatingthe second media resource request with a default queue.
 11. The methodof claim 10, further comprising associating a second media resource withthe default queue.
 12. The method of claim 1, further comprisingassigning the media resource request a place in the queue.
 13. Themethod of claim 12, wherein assigning the media resource request a placein the queue comprises determining whether the media resource requesthas a higher priority than any other media resource request currentlypending in the queue.
 14. The method of claim 12, wherein assigning themedia resource request a place in the queue comprises placing the mediaresource request in line behind previously-received media resourcerequests pending in the queue unless the media resource request has ahigher priority than one or more previously-received media resourcerequests pending in the queue.
 15. The method of claim 12, furthercomprising receiving an indication that the media resource requestshould be moved to a different place in the queue than the placecurrently assigned.
 16. The method of claim 1, further comprising:receiving an indication that the media resource is available; selectingfrom the queue a selected request to be serviced by the media resource;and servicing the selected request.
 17. The method of claim 1, whereinthe media resource request is associated with the queue based at leastin part on data included in the request.
 18. The method of claim 1,wherein the media resource request is associated with a set of data andthe media resource request is associated with the queue based at leastin part on the set of data to which the request relates.
 19. The methodof claim 1, wherein the media resource request is associated with aclient on which a set of data associated with the media resource requestis stored and the media resource request is associated with the queuebased at least in part on the client with which the media resourcerequest is associated.
 20. The method of claim 1, wherein the mediaresource request is associated with an application that generated themedia resource request and the media resource request is associated withthe queue based at least in part on the application which generated themedia resource request.
 21. A system for processing requests for mediaresources, comprising: a processor configured to: define a queue;allocate a media resource to the queue; and associate a media resourcerequest with the queue; and a memory configured to store the mediaresource request and other data associated with the queue.
 22. Acomputer program product for processing requests for media resources,the computer program product being embodied in a computer readablemedium and comprising computer instructions for: defining a queue;allocating a media resource to the queue; and associating a mediaresource request with the queue.